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Foreword 



Over the last 10 years the Center for Substance Abuse Treatment (CSAT) has 
accumulated a great deal of experience in substance abuse treatment evaluation implemented 
through coordinating centers, cross-site efforts, and national studies. The importance and value 
of integrating ongoing evaluation activity into a system for treating substance abuse problems is 
widely recognized. Also widely recognized, however, is that current evaluation-generated 
knowledge and practice are often under-utilized, due in part to the lack of an integrated approach 
to capturing information with which to measure and improve treatment effectiveness, efficiency, 
and performance. CSAT recognizes that such an integrated evaluation approach will more 
effectively support current and future knowledge generating activities. 

Based on a decade of evaluation experience, CSAT has developed the Integrated 
Evaluation Methods (lEM) Package, a series of conceptual and methodological applications, 
including concept papers, technical assistance materials, and analytic tools, to enhance CSAT- 
ftmded evaluation activities. Products in the lEM Package are organized within an evaluation 
framework constructed on the basis of accumulated experiences among internationally known 
treatment service evaluation professionals. Thus, the framework is based upon evaluation 
strategies, structures and approaches appropriate for substance abuse treatment evaluators and 
providers. The framework follows a standard set of evaluation activities: planning, selecting a 
design, developing data requirements and collection instruments, collecting and analyzing the 
data, and reporting the evaluation findings. (A summary description of the lEM Package is 
contained in Appendix A to this document.) 

This concept paper and its companion documents. Integrated Evaluation Methods: A 
Guide for Substance Abuse Treatment Knowledge-Generating Activities; Self-Adjusting 
Treatment Evaluation Model; Building Team Capability to Fully Implement and Utilize the Self- 
Adjusting Treatment Evaluation Model; Performance Measurement for Substance Abuse 
Treatment Services, and Client Levels of Functioning as a Component of Substance Abuse 
Treatment Services Evaluation present state-of-the-art conceptual models addressing issues 
related to coordination of treatment and evaluation activities, and integration of clinical, 
performance and evaluation information. Specifically, this concept paper identifies an approach 
that will enable the evaluation team to measure costs and to demonstrate the value of their 
services. 

Sharon Bishop 
Project Director 
NEDTAC 
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I. Introduction 



The Center for Substance Abuse Treatment (CSAT) supports the integration of ongoing 
evaluation within substance abuse treatment activities so as to demonstrate treatment service 
effectiveness and to improve treatment services and their outcomes. To this end, CSAT 
recommends the use of state-of-the-art evaluation methods and tools in planning, designing, and 
implementing treatment services evaluations. This document provides a discussion of the need 
for and types of cost analysis in the evaluation of knowledge-generating activities. 

Demonstrating and documenting the “efficiency” and/or “value” of treatment services are 
among the greatest challenges facing the substance abuse treatment field today. Put simply, 
efficiency/value is the ratio of outcomes to cost, and both types of data are necessary to 
effectively manage treatment services. While efficiency/value has always been important in 
managing a treatment system, this challenge is becoming increasingly acute. 

1. CONTEXT FOR THE ADDING “VALUE” TO CSAT DEMONSTRATIONS 
DOCUMENT 

CSAT’s major evaluation goals are to: (1) increase knowledge about substance abuse 
treatment services; (2) improve treatment services by applying knowledge gained through 
knowledge development and application (KD&A) activities; (3) develop analytic methods and 
approaches for use in knowledge-generating activities; and (4) develop substance abuse treatment 
analysis databases. To meet these goals, CSAT has been sponsoring KD&A initiatives including 
activities that focus on homelessness, marijuana use and treatment, managed care, women and 
violence, and opioid treatment, as well as the replicability of exemplary treatment approaches 
(e.g., methamphetamine treatment) and the evaluation of best practices for targeted populations 
(e.g., exemplary adolescent treatment). 

CSAT’s evaluation experiences have reinforced the fact that substance abuse treatment 
evaluation involves a standard set of tasks that generally occur in the following order: 

■ Planning the evaluation, which includes setting the evaluation goals and 
objectives that determine the overall parameters of the evaluation 

■ Selecting the evaluation design, which sets forth the overall strategy for 
establishing the evaluation questions, measurement approach, and generalizability 
of findings 
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■ Developing the data requirements, which flow from the evaluation questions 
and measures and include SDU, clinician, cost, and client data 

■ Developing data collection instruments, which are based on the data 
requirements and are developed or selected from a standard inventory of 
instrumentation 

■ Collecting the data, which includes the development of data management 
processes and tools including quality control procedures, and collecting the data 

■ Analyzing the data, which involves developing an analysis plan and conducting 
multiple levels of comparison; the analysis process is governed by the analysis 
plan and intended products and target audience(s) 

■ Reporting the evaluation findings, which includes evaluation knowledge 
dissemination and application within field. 

CSAT has directed the development of evaluation concepts, methods, and tools to support these 
evaluation tasks. The evaluation tasks and corresponding evaluation methods are summarized in 
Exhibit I, Appendix A. A full discussion of the CSAT evaluation analytic framework and the 
other evaluation concepts and tools (the Integrated Evaluation Methods Package), is presented in 
the concept paper: Integrated Evaluation Methods: A Guide for Substance Abuse Treatment 
Knowledge Generating Activities. The lEM package is referenced in Appendix A, and is being 
made available through the Caliber Associates NEDS contract Web site at http://neds.calib.com. 

2. IMPORTANCE OF COST ANALYSIS IN EVALUATIONS 

Rapid and dramatic changes are being made to the management, delivery, and financing 
of substance abuse treatment. These changes have strong implications for, and place significant 
demands on, the publicly financed substance abuse treatment system. Increasing importance is 
being placed on knowing the “cost” of treatment, while what actually constitutes “treatment” 
seems to be changing. 



The thesis of this paper is that cost analyses should receive much greater emphasis in the 
evaluation of substance abuse treatment service operations. Such information is invaluable right 
now. Moreover, reliable and rigorous methodologies for cost analyses are currently available. 
CSAT-funded substance abuse treatment providers must undertake cost analyses, as well as 
process and outcome evaluations, in order to gain information that will allow them to fully 
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capitalize on the significant investments that are being made in services in the name of 
generating and improving knowledge about substance abuse services. 



The rest of this document will expand upon 
these themes and identify an approach for 
incorporating cost information in substance abuse 
treatment evaluation activities. The primary 
emphasis will be given to a Uniform System of 
Accounts and Cost Reporting for Substance Abuse 
Providers, a treatment cost method developed for 
CSAT, which has already been tested on over 140 
treatment providers. The adoption and application of 
this cost method could potentially move CSAT 
evaluation activities and the treatment field forward 
significantly in the search for efficiency/value. 



3. ORGANIZATION OF THIS DOCUMENT 

The cost discussion is presented in the following 



To address efficiency/value, a 
meaningful evaluation must 
incorporate: 

■ Cost analysis 

■ Process evaluation 

■ Outcome evaluations. 

Like a three-legged stool, all 
three components are needed 
in order to have a secure place 
to sit. 



I 






sections: 



■ Section I, the Introduction, provides an overview of the paper 

■ Section II describes changes in the substance abuse treatment system that contributed 
to the need for cost benefit and cost effectiveness analyses 

■ Section III discusses cost benefit and cost effectiveness analyses 

■ Section IV presents an overview of the three levels of cost analyses specificity that 
can be incorporated into CSAT requirements, and briefly reviews the arguments for 
and against these approaches 

■ Section V presents a summary of the paper and a discussion of current and future 
applications of cost analysis 

■ Section VI summarizes cost analysis in relation to the Self-Adjusting Treatment 
Evaluation Model. 
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Appendix A contains a summary description of the Integrated Evaluation Methods package, of 
which this concept paper is one component; Appendix B is an explanatory note concerning the 
Self-Adjusting Treatment Evaluation Model (SATEM). 
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II. Evolution of Costs and the Substance Abuse 

Treatment System 



As noted in the previous text, cost analysis has always been an important part of the 
substance abuse treatment system, but it has become increasingly more important over the past 
several years. At the same time, the nature of cost analysis has changed significantly because of 
changes in the expectations (demands) of virtually all stakeholders. This evolution in the 
importance and nature of cost analysis demonstrates the impact that cost analysis had on Self- 
Adjusting Treatment Evaluation Model (SATEM) components that contribute to knowledge 
development and application. (See Appendix B.) 

While cost data have always been critical to the management of the substance abuse 
treatment system, they were never as central as they are now. The past 1 0 years have seen costs 
assume near primacy as a management concept, and old concepts and measurements of cost are 
giving way to new constructs. While old measurement approaches were not necessarily wrong, 
they were not adequate to address present or future requirements for fiscal accountability and 
improvement in treatment services. 

Stereotypes of the “old” and “evolving” system paradigms and their associated cost 
concepts are compared in Exhibit II- 1. This chart conveys in broad strokes how the demands for 
cost data have changed along with larger substance abuse treatment system changes. This 
comparison is not intended to suggest that the “evolving system” is superior to the “old system.” 
Rather, it attempts to suggest why and how the system is evolving and to indicate components of 
the system that can bear strengthening. 



Exhibit II-l 


Old and Evolving Cost Paradigms in Substance Abuse Treatment 


THE OLD PUBLIC SYSTEM 


THE EVOLVING COST SYSTEM 


Treatment Rationing 


Treatment Entitlement 


Maximize Outcome 


Minimize Cost 


System Expenditures 


Population Expenditures 


Provider Budgets 


Cost/Utilization per Beneficiary 


Treatment Slot Cost 


Cost per Treatment Episode 


Provider Expenditures 


Cost per Unit of Service 



Source: The Lewin Group. 
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The most fundamental change occurring in the system’s management is movement from 
treatment rationing to treatment entitlement. The public system has historically operated with a 
prospectively determined fixed annual budget, with an associated expected ability to pay for a 
certain amount of care. At the same time, policy makers have understood that this budget was 
not equal to the demand or need for care. There has been an acceptance of waiting lists, or 
queuing. Implicit in this system was the recognition that many of those who were wait-listed 
disappeared, which served to limit demand. Although resources per unit of treatment capacity 
(treatment slot) were carefully managed in this system, once a client occupied a slot, a provider 
could often retain clients in treatment for relatively longer durations to improve client outcomes. 
Providers were accountable for types and amounts of expenditures relative to current client 
enrollment, but had more discretion in terms of the mix of types of services delivered to clients. 

In contrast, the new cost environment is shifting to “treatment entitlement,” accompanied 
by strong pressures to keep costs down. Some would also suggest that financing and coverage of 
substance abuse treatment is moving toward “mainstreaming” with the rest of health care. Under 
such a scenario, an individual is entitled to treatment if “need” is demonstrated, given that the 
benefit has not been exhausted. (Ignore, for the moment, that substance abuse and mental health 
treatment benefits tend to have much lower “caps,” or limits, than somatic disorders.) Access to 
care becomes a major issue when there is entitlement. 

The recent over-riding focus of mainstream health care financing is on cost containment. 
Because the entitlement nature of insurance makes it somewhat difficult to limit initial access to 
care, there is more emphasis on controlling costs per treatment episode. This means that duration 
of care, service mix, and costs per unit of treatment service are more closely monitored and 
controlled. In this environment, there is often not as much data collected related to the quality of 
care, nor is there sufficient effort expended to relate quality to duration of care, specific types of 
services provided, or cost per unit of service. Reimbursement arrangements are increasingly 
shifting financial risks to providers, using competition among providers to control costs at a time 
when there is a lack of generally accepted and endorsed objective standards for quality. 

Treatment providers who have historically focused more on clinical management of their 
caseload, given a fixed budget, are now being driven to even greater efforts to economize. It is 
important to recognize that the cost and data requirements of the new environment are materially 
different from the old system. 
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One type of data that will be fundamental to the evolving substance abuse delivery 
system is efficiency/value data that contrast quality/outcomes to costs. Such data cannot be 
developed, however, without addressing the cost accounting changes in the new system. 



Cost accounting has proven 
challenging to many existing publicly 
financed treatment providers. The central 
points of this paper are: 

■ There are methodologies and 
tools to measure costs 

■ Incorporating these tools in substance abuse treatment evaluations would yield more 
comprehensive results 

■ These applied methods can be used as tools to assist the treatment services 
community to operate in the evolving environment. 

The other components of the efficiency/value equation — quality/outcomes — are already being 
addressed in other facets of the evaluation designs CSAT is advocating through the SATEM. 



Efficiency/value data that contrast | 

quality/outcomes to costs is | 

fundamental to the evolving i 

substance abuse treatment delivery | 

system. I 
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III. Types of Cost Studies 



Several different types of cost studies are often discussed almost interchangeably despite 



the fact that they are fundamentally different 
from each other: 

■ Cost analysis (CA) 

■ Cost effectiveness analysis (CEA) 

■ Cost benefit analysis (CBA). 

These analyses are actually quite different from 
the perspective of economics and in terms of 
their rigor and difficulty. Cost analysis (CA) 
simply attempts to estimate the expenses of 
delivering the services. Equivalent cost data for 
different service providers allow comparisons to 
be made of the relative expenses for treating 
substance abusers, and even the relative cost of 
specific components of service. However, CA 
by itself yields no insights into the quality or 
effectiveness of the care that is delivered. 

Efficiency/value conclusions can only be 
generated from cost effectiveness and cost 



Types of Cost Studies 

Cost analysis (CA) attempts to 
estimate the expense of 
delivering the services, but 
provides no information 
regarding quality or 
effectiveness. 

Cost effectiveness analysis 
(CEA) uses a single outcome 
measure for comparison. In such 
analyses, the outcome measure 
is contrasted to the cost of 
treatment, and ratios of outcome 
to cost are compared for each 
alternative considered. 



Cost benefit analysis (CBA) is 
used when there are multiple 
outcome measures, and 
efficiency comparisons across 
different outcomes yield different 
conclusions. 









benefit analysis (CEA and CBA, respectively). 

The explicit objective of such analyses is to analyze whether resources (funds) are being used 



more or less efficiently in different treatment settings. These analyses can be framed as 



comparing: 



■ Treatment (of some type) versus no treatment 

■ Treatment A versus treatment B (versus C, D, etc.) 

■ No treatment versus treatment A versus treatment B (versus C, D, etc.). 

Thus, in a fundamental respect, CEA and CBA are similar in their purpose, but they differ in the 
complexity of their assessment of outcomes. 
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Cost effectiveness studies only use a single outcome measure for comparison, such as the 
proportion of clients abstinent or the proportion with no arrests at 6 or 12 months post treatment. 
The outcome measure is contrasted to the cost of treatment, and the ratios of outcome to cost are 
compared for each alternative examined in the study. If two alternatives yield the same outcome, 
then the less expensive is considered more efficient. If two alternatives cost the same, but one 
has a better outcome, then it is more efficient. If an alternative has both a better outcome and 
costs more, then policy makers must evaluate the trade-off and decide if they are willing to pay 
more to get more. 

The primary weakness of CEA is that only a single outcome can be examined at a time, 
although it is possible to make multiple CEA comparisons (using a number of different outcome 
measures) across the alternatives. If all of the CEA comparisons support the same efficiency 
conclusion there is no problem. However, the results of CEA can be difficult to interpret when 
there are multiple outcomes of interest (e.g., one alternative performs better on some measures, 
and (an)other altemative(s) perform(s) better on other outcomes). 

Cost benefit analysis (CBA) is necessary when there are multiple outcome measures, and 
efficiency comparisons across different outcomes yield different conclusions. The objective of 
CBA is essentially to weight the various outcome dimensions into a single outcome (benefit) 
index, which can then be contrasted to the costs in order to judge relative efficiency. 

Typically such benefits measures are constructed 
using economic (dollar) values for each outcome measure 
and summing across the measures to generate a composite 
benefit measure. Economic values, thus, constitute the 
weighting central to the “index.” Analysts skilled in 
economic theory and measurement construct such 
“indices.” Treatment alternatives can then be compared by 
contrasting the ratios of benefits to the dollar cost of the treatment. The implication of a 
treatment alternative having a superior (higher) ratio of benefits to costs is that greater aggregate 
benefits can be achieved by investing scarce treatment dollars into that alternative. 

Both cost analysis and outcome analysis are required to perform efficiency/value studies, 
be they CEA or CBA studies. Again, other CSAT documents such as the Self-Adjusting 
Treatment Evaluation Model address study design and outcome analysis. The remainder of this 
paper concentrates on the analysis of costs and the options that may be employed for this 
purpose. 

O 
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IV. Toward a Design for Cost Analysis for CSAT 
Demonstrations and Knowledge-Generating Activities 



The CSAT Knowledge Development and Application (KD&A) programs, affiliated 
cross-site efforts, and national evaluation study designs present a strong opportunity to advance 
the methodology and practice of cost analysis for substance abuse treatment. Treatment 
demonstrations such as those supported by CSAT’s KD&A programs are explicitly intended to 
advance knowledge that can promote the effectiveness and efficiency of the nation’s substance 
abuse treatment system. To date, much more emphasis has been placed on effectiveness 
(through process and outcome evaluations), however, than on cost and/or efficiency or value. It 
is proposed that treatment demonstrations should: 

■ Analyze both treatment effectiveness (outcomes and process) and costs 

■ Examine both the total cost of care and of specific components of care (particularly of 
“interventions” that are being demonstrated) 

■ Use consistent and comprehensive cost methodologies across CSAT-funded treatment 
providers. 

The need to articulate an evaluation approach to cost analysis is driven by the simple fact that 
CSAT and the treatment services evaluation field currently can present very little 
methodologically comparable cost data related to demonstrations funded to date. The lack of 
cost data is true for all types of treatment services that have been supported under CSAT 
demonstrations: 

■ Systems enhancements (Target Cities, Linkage and Outreach, Managed Care) 

■ Services for specific populations (e.g.. Women and Children, Correctional Treatment, 
Adolescent Treatment) 

■ Particular components or interventions (HIV services, mental health, employment, 
marijuana and methamphetamine interventions). 

The CSAT approach to evaluation and evaluation requirements has clearly undergone a 
significant shift as evidenced by: 

■ Mega evaluations such as the National Treatment Improvement Evaluation Study 
(NTIES) 



O 
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■ Individual evaluations 

■ Requirements that each provider perform process and outcome evaluation 

■ Providers being given minimum specifications for evaluation design and content 

■ Design and implementation of cross-site evaluations with a uniform design and 
common data elements for all projects in a demonstration 

■ Development of minimum data sets and a common analytic framework and an 
integrated evaluation methodology for use across CSAT program areas. 

The enhanced emphasis on knowledge generation that is embodied in the CSAT knowledge 
development and application initiatives, combined with a cost analysis methodology yield the 
bundle of analytic methods necessary to incorporate cost data into the evaluation of treatment 
demonstrations. 

The initial CSAT approach was to undertake the NTIES mega evaluation cutting across 
most of the demonstrations funded in FY 90-91 . While this yielded consistent data about 
organizations that had their clients sampled for inclusion in NTIES and produced reliable data 
about these CSAT program areas, it was not actually designed to yield data or inferences about 
individual providers or programs not included in the sample (e.g., residential women and their 
children, AIDS outreach, etc.). These types of mega evaluation studies are invaluable, but cannot 
address all of the evaluation needs in a field as diverse as substance abuse treatment. Another 
limitation to the approach utilized in NTIES is the significant time lag related to the design and 
implementation of a large nationwide study. In contrast, individual providers can move more 
quickly in designing and executing their particular evaluations, and opportunities for high quality 
evaluation are significantly improved if sites employ consistent study designs and collect 
consistent data. 

Realizing that an NTIES would not necessarily address the most important evaluation 
questions for individual CSAT demonstrations over time, CSAT also began to require 
evaluations (process and outcome) of individual providers of treatment services in FY 93. The 
experience of the past several years, however, has shown that non-specific requirements for 
providers to perform process and outcome evaluations has been of limited utility. First of all, 
process and outcome evaluation requirements have generally been construed to not include cost 
analysis (although cost analysis could be performed under the process analysis rubric). Second, 
most local evaluators and managers who asked about evaluation requirements are unequivocal in 
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and Knowledge-Generating Activities 



their desire to have early and more specific guidance from CSAT about what activities to 
evaluate and how to evaluate them. CSAT has in fact moved in the direction of incorporating 
such guidance or standards in the initial Government Guidance for Applicants (GFAs). Even 
with such requirements, however, the findings from individual grant activities are not 
comparable to each other due to differences in designs. To address this, in FY 95 CSAT began 
to establish specific requirements for evaluation designs that CSAT-funded treatment service 
providers were responsible for incorporating into their “local” evaluations. 

This paper articulates the rationale for expanding this evaluation strategy to include cost 
analysis, and hopefully promotes performing cost effectiveness and cost benefit studies. The 
following sections describe three approaches to cost analysis which can be incorporated into 
evaluation requirements, and briefly reviews the arguments for and against these approaches. 
The three approaches for cost analysis being considered by CSAT are: 

■ Cost analysis with limited CSAT specifications regarding approach 

■ Analysis with specialized CSAT protocols for each CSAT-funded treatment provider 

■ Analysis with same CSAT specifications applied across all CSAT-funded treatment 
providers. 

These three approaches reflect different levels of CSAT specificity for the CSAT analysis that 
would be conducted by CSAT-funded treatment service providers. 

1. LIMITED COST SPECIFICATIONS 

Under this approach, CSAT would present substance abuse treatment service providers 
with general guidelines for the analysis of the costs of their activities. This approach would 
direct cost analysis of services supported by CSAT. Moreover, these services would need to be 
separated and distinguished from services not receiving CSAT support. For example, if a 
provider receives support to add child care services in order to enable women to participate in 
treatment, the cost analysis should produce estimates of the expense of those services, 
incorporating and reporting data on the rate of utilization of the services as well as on the inputs 
necessary to deliver child care services (e.g., staffing, space, utilities, supplies). Other analyses 
could examine the costs involved with delivering other services to a parent; however, the main 
focus would be on the CSAT-funded intervention. 
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Toward a Design for Cost Analysis for CSAT Demonstrations 
and Knowledge-Generating Activities 



The use of limited specifications has the primary advantage of giving treatment providers 
maximal flexibility to address unique aspects of their services. There are, however, significant 
disadvantages to this approach. First, it will be impossible to make comprehensive and valid 
comparisons across demonstration activities and to discern if various approaches to the same 
service/intervention are more or less expensive or more or less efficient (comparing outcome to 
cost). Second, it severely limits the ability to generalize findings and to inform the field about 
the general cost or efficiency of a type of service. Variations in costs may be artifacts of 
different cost accounting methodologies rather than actual differences. Even so, an important 
first step is to incorporate cost data in the evaluation strategy of substance abuse treatment 
demonstrations. 

2. SPECIALIZED COST SPECIFICATIONS FOR A GIVEN PROGRAM AREA 

A step beyond the limited specification approach is to develop specialized specifications 
for each CSAT program area. Again, this methodology would require cost analysis to focus on 
and distinguish the particular 
services/interventions that are 
supported by CSAT, and analyze the 
other components linked to the 
service. Unlike the limited 
approach, this methodology would 
define specific cost accounting 
methods for estimating costs, and 
would also utilize appropriate and 
standardized accounting principles. 

The primary advantage of 
this approach is that it would 
promote detailed and specific 
analyses of all of the treatment 
service providers within a program 
area, and would provide particular 
insights into the services/ 



Advantages and Disadvantages of Different 
Cost Analyses 

Level of CSAT Specification for: 



Type 


Advantage 


Disadvantage 


Limited 


Maximum 
flexibility for 
unique needs 


Inability to 
compare or 
generalize 


Specialized 


Comparison of 
similar projects 


Inability to 
compare 
across 
providers or 
generalize 


General 


Can be utilized 
across CSAT 
program areas 


Requires 

related 

expertise 



I 
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Toward a Design for Cost Analysis for CSA T Demonstrations 
and Knowledge-Generating Activities 



interventions that are the focus of the CSAT program area. This will also begin to put into place 
knowledge-generating capability. 

This approach also has disadvantages. First, the specialized nature of the cost 
methodology, while allowing rigorous comparisons across the local implementation sites in the 
program areas, might not be comparable to or useful for broader purposes. By focusing on a 
particular CSAT program area (e.g.. Residential Women and Children, Criminal Justice 
Network, HIV Outreach, or Managed Care), more attention may be given to aspects of cost that 
are less important in other types of treatment settings (e.g., prenatal and child care). Thus, it may 
not be possible to make rigorous comparisons across CSAT program areas, or to generalize 
programs in the general system unless extensive work is done in the design stage to ensure 
comparability. 

Finally, it may simply require too much time to initiate development of a rigorous cost- 
estimation methodology that meets highly specialized needs for one CSAT program area, but 
also maintains comparability to other program areas. In terms of methodology development, this 
approach would be very costly. This may also entail some duplication of effort, unless there is 
close coordination across the development efforts. 

3. GENERAL COST SPECIFICATIONS ACROSS PROGRAM AREAS 

The most productive approach is to develop a cost estimation methodology that is 
appropriate for utilization within all CSAT program areas. Such a cost methodology has been 
developed at the direction of CSAT’s Program Evaluation Branch. Known as the “Uniform 
System of Accounting and Cost Reporting for Substance Abuse Treatment Providers,” it was 
developed for the Center for Substance Abuse Treatment by accounting experts at Capital 
Consulting Corporation in conjunction with a national expert panel. The Uniform System of 
Accounting can serve as the basis for cost analyses performed by service providers as part of the 
CSAT knowledge-generating activities. The Uniform System of Accounting applies standard 
accounting principles and techniques to generate a cost estimation methodology specifically for 
substance abuse treatment services (Capital Consulting Corporation, 1993, 1994, and 1995; 
Lewin-VHI, Inc., 1995). The Uniform System was developed for CSAT with the intention of 
providing a generally available and applicable standard cost estimation methodology for the 
substance abuse treatment field. Such a tool facilitates valid and reliable comparisons of costs 
across providers, as well as assists individual providers in estimating the cost of their services 
(and service components). This would give providers accurate cost information with which to 
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Toward a Design for Cost Analysis for CSAT Demonstrations 
and Knowledge-Generating Activities 



negotiate treatment contracts with managed care and public treatment authorities as well as 
provide cost data to assess and improve their own services. 

This methodology has been piloted to estimate the costs for over 140 substance abuse 
treatment service delivery units (SDUs) across the nation since 1992. Most of these have been 
treatment providers funded by CSAT, across many different program areas. Types of services 
analyzed have included day treatment, outpatient, community-based residential, hospital-based 
detoxification and rehabilitation, and methadone maintenance (Capital Consulting Corporation, 
1994; Lewin-VHI, Inc., 1995). The methodology 
has been applied to “systems,” such as Target 
Cities demonstrations and has proven amenable 
to analyses of centralized system components 
such as central intake units. 

The initial step in the approach of the 
Uniform System is to acquire comprehensive data 
about expenditures for substance abuse service 
providers by SDU, including all of the general 
categories of expenses which are presented in 
Exhibit IV-1 (Capital Consulting Corporation, 

1993). Note that these are categories of expenses 
and that under each category there may be 
numerous items. For example, administration costs include the services of various different 
types of personnel, equipment rental/service/supplies, various professional services (e.g., legal, 
bookkeeping), telephone, insurance, and other. It is also necessary to count direct salaries, fringe 
benefits, and payroll taxes for all personnel. 



The Uniform System of Accounting 
requires: 

■ Comprehensive data about 
expenditures by SDU 

■ Unit cost estimates for all 
services 

■ Allocation of cost by service 






Unit cost (service cost divided 
by number of units). 
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Toward a Design for Cost Analysis for CSA T Demonstrations 
and Knowledge-Generating Activities 



EXfflBITlV-1 

Types of Expenses of Substance Abuse Treatment Providers 


Administration 


Psychiatrist 


Facility and Grounds 


Psychologist 


Dietary ^ 


Social Worker 


Laundry 


Certified Addiction Counselor 


Housekeeping 


Vocational Therapist 


Medical Care 


Recreational Therapist 


Laboratory 


Other Therapists 


Depreciation, Rent, and Interest 





Source: Capital Consulting Corporation, 1993. 



The ultimate product of the Uniform System is unit cost estimates for 13 broad types of 
services that comprise virtually all of the activities of substance abuse treatment providers. 

These services are identified and defined in Exhibit IV-2. Note that each “type” of service may 
actually be comprised of a number of related types of services, such as medical/diagnostic 
services, which could well include tests for TB, hepatitis, and pregnancy, as well as many others. 

Also, for each type of service, a unit of service measurement is defined, primarily in order 
to allow costing per unit of service. The most typical unit of service is the number of clients 
treated by the provider over the time period, although other important units are the number of 
individual and group counseling hours delivered. 

Clearly, the objective of the Uniform System is to generally characterize the level of 
resources being put into discrete types of services. This methodology could be extended in order 
to define more narrowly precise types of services, and to estimate costs for those services. The 
national expert committee judged that the 13 types of services would constitute a solid initial 
typology that would start the field moving toward greater cost awareness. 
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Toward a Design for Cost Analysis for CSAT Demonstrations 
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Substance Abuse Provider Unit Services 


TYPE OF SERVICE 


UNIT OF SERVICE 


Initial Assessment 


Number of Assessments 


Medical Examination 


Number of Clients 


Psychosocial Examination 


Number of Clients 


Individual Counseling 


Number of Counseling Hours 


Group Counseling 


Number of Counseling Hours 


HIV Testing/Counseling 


Number of Clients Counseled 


Medical/Diagnostic Services 


Number of Medical Examinations 


Housing and Food Services 


In-residence Days 


Clinical Case Management 


Number of Cases 


Networking/Outreach 


Number of Contacts 


Child Care Services 


Number of Children 


Staff Education 


Number of Hours 


Client Education 


Number of Cases 


Total 


Number of Clients 



Source; Capital Consulting Corporation, 1993. 



The core of the Uniform System is comprised of a large number of allocation rules for 
translating expenses into unit costs for each type of service. This may be simplistically 
represented by Exhibit IV-3, which shows that the Uniform System ultimately translates 
expenses by category into expenses by type of service. Specifically, a provider starts with total 
expenses by category (listed across the top of the table) and allocates these within a column 
across the respective services (listed down the side of the table). Once total expenses by type of 
service are summed, this is divided by the number of units of that type of service that have been 
delivered during the period being studied. A version of this approach is being incorporated into 
the revised financial pages of CSAT’s quarterly progress financial reporting formats for the 
demonstration projects. 
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Exhibit rV-3 

Cost Analysis: Translating Expenses into Unit Costs 


CATEGORIES OF EXPENSES 
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TYPES OF SERVICES 


Initial Assessment 


1 

Medical Examination 


Psychosocial Examination 


Individual Counseling 


Group Counseling 


HIV Testing/Counseling 


Medical/Diagnostic Services 


Housing and Food Services 


Clinical Case Management 


Networking/Outreach 


Child Care Services 


Staff Education 


Client Education 


Total Expenses by Type 
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Toward a Design for Cost Analysis for CSAT Demonstrations 
and Knowledge-Generating Activities 



An example of the final product of such a cost analysis methodology for a particular 
provider using the Uniform System is presented in Exhibits IV-4 through IV-6. The sample 
provider delivered several types of services (or operated multiple service delivery units) to 
homeless substance abusers, including residential care, outreach services, and transitional 
housing for treatment completers. The provider had total expenses of $380,000 for 1 993. 
Application of the Uniform System of Accounts allocated these total expenses into $240,000 for 
the residential treatment phase, $35,000 for outreach services, and $95,000 for transitional 
housing services. Further analysis broke those costs down by the 13 types of services, and 
calculated the costs per unit of service. 

Note that in this multi-service organization there were expenses for networking/outreach 
in both the designated “outreach” service and in the residential service delivery unit, where it was 
estimated that counselors on average put 15 minutes per week into this activity. Thus, outreach 
accounted for $25,000, or about 10 percent of the total budget, and was allocated across 57 total 
clients at a cost of $437 per client. In contrast, the dedicated outreach services had a total budget 
of $35,000, and accomplished 6,800 contacts over the year, for a cost per contact of about $5. 

HIV testing and counseling was delivered at the residential unit ($1,700 per year, or $29 
per client); but, if it was performed as part of outreach services, it could not be broken out for 
purposes of cost allocation. Housing services constituted the largest portion of total costs, about 
55 percent ($30.00 per client night) in the residential setting, and 99 percent of costs ($1 1.31 per 
night) in the transitional setting. 

The Uniform System has been used to develop cost estimates for over 140 substance 
abuse service delivery units since 1992. During this time the allocation methodology has been 
tested and refined to the point that the methodology appears to be ready for broader application 
to CSAT-sponsored substance abuse treatment knowledge-generating activities. 
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Exhibit IV-6 

Cost Profile OF a Traivsitional Living Service Delivery Unit of a Multi-modality Provider 

Locatioii: Urban Transitionai Living for the Homeless 

FYE: December jl, 1993 Program Cost Profile 


PER CLIENT 

average 


000$ 


000$ 


000$ 


000$ 


000$ 


000$ 


oo 

00 


$2,270.52 


000$ 


000$ 


000$ 


000$ 


000$ 


000$ 


$2,275.40 


UNIT COSTS 


000$ 


000$ 


000$ 


000$ 


000$ 


000$ 


OO 

OO 


SII.3I 


000$ 


oo;o$ 


000$ 


000$ 


000$ 


000$ 


fO 
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42 


8,429 


o 


O 


o 
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8,429 


PROGRAM 

COST 


$0 


$0 


$0 


$0 


$0 


$0 


$205 


$95,362 


$0 


$0 


$0 


$0 


$0 


$0 


$95,567 


UNIT MEASUREMENT 


Number of Assessments 


Number of Clients 


Number of Clients 


Number of Counseling Hours 


Number of Counselor Hours 


Number of Clients Counseled 


Number of Medical Examinations 


In-residence Days 


Number of Cases 


Number of Contacts 


Number of Children 


Number of Hours/Clients 


Number of Cases 


Number of Clients 




UNITS OF SERVICE 


Initial Assessment 


Medical Examination 


Psychosocial Evaluation 


Individual Counseling 


Group Counseling 


HIV Testing/Counseling 


Medical/Diagnostic Services 


Housing and Food Services 


Clinical Case Management 


Networking/Outreach 


Child Care Services 


Staff Education 


Client Education 


Client Transportation 


TOTAL 
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V. Summary 



Evolution of cost analysis in substance abuse treatment is almost a perfect example of 
how the Self-Adjusting Treatment Evaluation Model (SATEM) contributes to Knowledge 
Development and Application (KDA). Cost analysis should be a learning community activity. 
Cost analysis generates knowledge that is useful for service providers for system decision- 
making and for policy-making. To the extent that such knowledge actually affects operations or 
policies, it provides for self-adjustment. In order to assess the value of services provided by 
substance abuse providers, cost evaluation data are now being used in conjunction with 
performance evaluation data. Useful cost analysis requires integration of evaluation, clinical 
knowledge, and multidisciplinary training. 

Cost analysis is necessary if treatment evaluations are going to examine efficiency or 
value in substance abuse treatment. Cost analysis data in conjunction with outcome data will 
make it possible to perform cost effectiveness and cost benefit analyses. There are very strong 
arguments for this. 

Between the three options outlined, experience has shown that service providers and 
evaluators do not prefer the “limited specification” approach. If anything, they have shown a 
strong preference for clear standards for performing evaluations. This preference is likely to be 
even stronger for cost analyses than it is for process and outcome evaluations, since many 
treatment providers perceive themselves or their evaluators as having the relevant expertise to 
conduct cost studies. Further, there are few models of cost analysis from which they can draw. 

The “Uniform System of Accounts” offers a methodologically sound model for 
performing cost analyses that are directly applicable to substance abuse treatment and related 
services. The model was explicitly designed to yield cost data that would be highly relevant to 
understanding how substance abuse treatment providers use their resources. Thus, it is able to 
support CSAT program areas that wish to examine these matters in a more rigorous manner. 

In its current form, the Uniform System can be applied to estimate costs of CSAT- 
supported program areas, as well as to estimate or break out the costs of particular 
services/interventions. The Uniform System can be adopted to meet the needs of a specific type 
of service or intervention, or to address a specific services research/evaluation question or focus, 
as demonstrated by its application in the Wrap Around Services program and the national cross- 
sites. 
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VI. Conclusion 



This document discusses the need to conduct cost analysis when evaluating substance 
abuse treatment services. It presents three cost analysis strategies for CSAT demonstration 
treatment service providers. The paper also shows how CSAT’s “Uniform System of 
Accounting and Cost Reporting for Substance Abuse Treatment Providers” can be used to 
develop cost analysis data. Full implementation of the Self-Adjusting Treatment Evaluation 
Model requires the use of cost analysis in combination with performance measurements, building 
of team capability, and assimilating CSAT’s integrated methodology. 

Adding "Value ’’ To CSAT Demonstrations: The What, How, and Why of Cost Analysis 
provides: 



■ A discussion of the evolution of the importance of “cost” in the substance abuse 
treatment system 

■ A discussion of the types of cost studies, including cost analysis, cost-effectiveness 
analysis, and cost benefit analysis 

■ A discussion of the types of cost analysis designs for CSAT-funded substance abuse 
treatment activities 

■ A discussion of the “Uniform System of Accounting and Cost Reporting for 
Substance Abuse Treatment Providers.” 



It is hoped that this information will enable individual service providers and the field to better 
implement the Self-Adjusting Treatment Evaluation Model in order to ensure continuous 
knowledge development and improvement of substance abuse treatment services. 
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Appendix A: 

Integrated Evaluation Methods Package: 

A Guide For Substance Abuse Treatment 
Knowledge-Generating Activities — Executive Summary 

Since its inception, the Center for Substance Abuse Treatment (CSAT) has provided 
Federal leadership to improve substance abuse treatment accessibility, effectiveness, and 
efficiency. CSAT’s mission and activities have evolved from directly supporting treatment 
services to supporting knowledge-generating activities. This evolution is evident in the current 
Substance Abuse and Mental Health Services Administration policy on evaluation as described 
in Evaluation Policy, SAMHSA, 1995. 

The need for an integrated model of evaluation and planning at SAMHSA is presented in 
“Evaluation in the Substance Abuse and Mental Health Services Administration,” Evaluation 
and the Health Professions, by Marsh, Jansen, Lewis, & Straw, 1996. CSAT also supports site- 
specific, cross-site, and national evaluations that have provided experience with a wide array of 
evaluation design and implementation methods. These experiences further supported the need 
for an integrated evaluation strategy and led to the development of a comprehensive set of 
evaluation products, including concept papers, technical assistance (TA) materials, and analytic 
tools. Collectively, these products are referred to as the Integrated Evaluation Methods (lEM) 
Package. The lEM Package organizes these products within an evaluation framework that is 
designed to support CSAT knowledge development and application goals. The evaluation 
framework itself was constructed on the basis of accumulated experiences among internationally 
known treatment service evaluation professionals. The lEM Package reflects and incorporates 
evaluation experiences gained over the past decade. 

Evaluation Framework and the Integrated Evaluation Methods Package 

National evaluation experiences have reinforced the fact that substance abuse treatment 
evaluation involves a standard set of tasks that generally occur in the following order: 

■ Planning the evaluation/knowledge-generating activities, which includes selecting 
the substance abuse treatment issue, identifying the theoretical foundation for the 
intervention, determining knowledge development program goals and implementation 
approach, and setting the evaluation goals and objectives that determine the overall 
parameters of the evaluation 
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■ Selecting the evaluation design, which sets forth the overall strategy for establishing 
the process and outcome evaluation questions, measurement approach, and 
generalizability of findings 

■ Developing the data requirements, which flow from the evaluation questions and 
measures and include: SDU, clinician, cost, and client data 

■ Developing data collection instruments, which are based on the data requirements 
and are developed or selected from an integrated inventory of instrumentation 

■ Collecting the data, which includes developing data management processes and tools 
(including quality control procedures) and conducting the data collection activities 

■ Analyzing the data, which involves multiple levels of comparison and is governed 
by an analysis plan 

■ Reporting the evaluation findings, which includes evaluation knowledge 
dissemination and application within the field. 

The evaluation process outlined above provided a framework for the development of products 
related to these evaluation concepts and methods. Taken together, those products comprise the 
lEM Package. 

Integrated Evaluation Methods Products 

CSAT requested the development of a series of evaluation concept papers, TA materials, 
and tools to support and operationalize each phase in the evaluation of substance abuse treatment 
knowledge-generating activities. These items are included in the lEM Package. The concept 
papers are based on theoretical evaluation research constructs that have been adapted to 
substance abuse treatment services evaluation and knowledge-generating activities. The concept 
papers primarily support the evaluation planning phase and address such topics as the self- 
adjusting treatment evaluation model, cost analyses, and performance measurement. The TA 
materials and tools include specific evaluation methods that have direct applicability to substance 
abuse treatment knowledge-generating activities. The concept papers and TA materials that 
constitute the lEM Package are listed and briefly described in Exhibit I. 
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EVALUATION 

FRAMEWORK 


INTEGRATED EVALUATION METHODS PRODUCTS 


1. Planning the 
evaluation/ 
knowledge- 
generating 
activities 


■ Integrated Evaluation Methods; A Guide for Substance Abuse Treatment 
Knowledge Generating Activities: Concept paper that describes the development of an 
evaluation framework, evaluation concepts, and TA materials to support the framework. 

■ Self-Adjusting Treatment Evaluation Model; Concept paper that describes an 
approach for integrating evaluation findings within treatment operations so as to adjust 
and improve service delivery. 

■ Building Team Capability to Fully Implement and Utilize the Self-Adjusting 
Treatment Evaluation Model: Concept paper to assist treatment providers in building 
capabilities to integrate the self-adjusting treatment model within day-to-day operations 
and service delivery. 

■ Adding “Value” to CSAT Demonstrations: The What, How and Why of Cost 
Analysis: Concept paper on the need for and types of cost analyses for CSAT 
demonstrations and knowledge-generating activities. (The Lewin Group) 

■ Performance Measurement for Substance Abuse Treatment Services: Concept 
paper about the increasing importance of provider performance measurement and 
analyses and an explanation of the case-mix adjustment methodology. 

■ Client Levels of Functioning as a Component of Substance Abuse Treatment 
Services Evaluation: Description of the rationale and methods for assessing client level 
of functioning and recommended core LOF data elements that could help to measure the 
effectiveness of treatment services received. 

■ Substance Abuse Treatment Evaluation Policy Notebook: These materials are aimed 
at facilitating understanding of the SAMHSA policy for evaluation and federal 
regulations on client confidentiality and assisting evaluators to meet CSAT evaluation 
requirements. 

■ Substance Abuse Treatment Evaluation Resource Notebook: The notebook contains 
evaluation bibliographies and listings of organizations, hot lines, on-line data bases, and 
contact information for obtaining assistance in evaluating treatment services. 


2. Selecting the 
evaluation design 


■ A Guide to Process Evaluation for Substance Abuse Treatment Services: TA tool 
presenting purposes of process evaluation and the application of process evaluation 
methods to single site and multi-site treatment services. 

■ Using Logic Models in Substance Abuse Treatment Evaluations: TA tool describing 
logic model purposes and techniques for designing and planning the evaluation of 
treatment services. 

■ A Guide to Selecting an Outcome Evaluation Design for Substance Abuse 
Treatment Evaluations: TA tool describing overall strategies for developing 
evaluation questions, measurement, controls, validity/reliability, sampling, design 
effects, and generalizability of findings. (Battelle) 



gj^Q:SAT\CTRT_END\IEM\CONCEPT\VALUEADD\DATA\COSTPPR.WPD 



BEST COPY AVAILABLE 



39 



NEDTAC, Page A-3 



Exhibit I (Continued) 
Evaluation Framework and Integrated 
Evaluation Methods Package 



Appendix A 



EVALUATION 

FRAMEWORK 


integrated evaluation methods PACKAGE 


3. Developing data 
requirements 


■ Minimum Evaluation Data Set (MEDS): Core Data Lists: TA tool for developing a 
uniform set of variables and response categories for the service delivery unit (SDU), 
clinician, cost, and client evaluation measures. 

■ Substance Abuse Treatment Cost Allocation and Analysis Template (SATCAAT): 
User manual to analyze treatment costs by unit of service for an SDU. (Capital 
Consulting Corporation) 


4. Developing data 
collection 
instruments 


■ Substance Abuse Treatment Services Evaluation Data Collection Instruments: Data 
collection instruments that fully incorporate the MEDS and that have been field tested 
for validity and reliability, as follows: Service Delivery Unit (SDU) Description; 
Clinician Background and Practice Survey; protocols to collect Adult, Adolescent and 
Child (in treatment with parent) Client Data at Intake, During Treatment, at Treatment 
Discharge and Post Treatment; Adult and Adolescent Record Extraction forms; and a 
section on protection of human subjects and informed consent. 


5. Collecting the 
data 


■ Staying In Touch: A Fieldwork Manual of Tracking Procedures for Locating 
Substance Abusers for Follow-up Studies (UCLA): User manual to establish and 
implement client follow-up data collection systems and procedures. 

■ Strategies for Follow-up Tracking of Juvenile, Homeless, and Criminal Justice 
System-Involved Substance Abusers: Overview and Bibliographies, 1990-1998: 
Description of tracking techniques used to increase response rates for follow-up 
interviews with homeless and Juvenile/criminal Justice involved substance abusers. 


6. Analyzing the 
data 


■ A Guide to Substance Abuse Treatment Evaluation Data Analysis: Recommended 
methods and procedures for analyzing process, SDU, clinician, cost, and client 
evaluation data. 


7. Reporting the 
evaluation 
findings 


■ Substance Abuse Treatment Evaluation Product Outlines Notebook: Compendium 
of outlines for evaluation products including evaluation plans, interim evaluation reports, 
final evaluation reports, replication studies, case studies, and ethnographies. 
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CSAT Evaluation “Stakeholders” 

Evaluation “stakeholders” are individuals, groups, or organizations that have a significant 
interest in how well a program or activity functions. (See P.H. Rossi, H.E. Freeman, & M.W. 
Lipsey, Evaluation: A Systematic Approach, 6th Edition, 1999.) Within the context of the lEM 
Package, CSAT evaluation stakeholders include CSAT senior managers, CSAT project officers, 
and CSAT grantees and contractors including treatment service providers, coordinating centers, 
study sites, site-specific evaluators, and national evaluators. 

Utility of the lEM Package for CSAT Evaluation Stakeholders 

While the conceptual and TA materials were developed from the perspective of the site- 
specific and multi-site evaluator, the concepts and TA tools have important utility for CSAT 
managers, project officers, and treatment service providers. The stakeholder’s position 
determines the perspective and utility of the lEM Package concepts and tools. For example, a 
CSAT senior manager can use the lEM Package to acquire a comprehensive evaluation context 
for planning and funding the knowledge-generating activities, the project officer can use the lEM 
Package to ensure that GFA/RFP applications are complete and include a full complement of 
design, execution, and product components, and the site-specific and multi-site evaluators can 
use the lEM Package to ensure that evaluation designs, data collection plans, data analyses, and 
product development have a consistent evaluation framework and compatible data across 
program areas. The suggested utility of the lEM Package for CSAT evaluation stakeholders is 
summarized in Exhibit II. 
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Exhibit II 

Utility of IEM Package for CSAT Evaluation Stakeholders 



STAKEHOLDERS 


ROLES AND RESPONSIBILITIES 


IEM PACKAGE UTILITY 


SENIOR 

MANAGERS 


■ Policy development 

■ Issue identification for KD&As 

■ Grant/contract funding decisions 

■ Overall program management 

■ Sustainability 

■ Dissemination 

■ Long-term strategic planning 

■ Program designs 

■ KA activities 


■ Comprehensive evaluation framework 

■ Comprehensive evaluation components 

■ Roles and responsibilities for local/national 
evaluators as well as CSAT/grantee staffs 

■ Guidance for evaluation designs and 
products 

■ Standardized evaluation measures 

■ Logic models for program and evaluation 
design 


PROJECT 

OFFICERS 


■ GFA/SOW development 

■ Grant/contract application review 

■ Grant/contract monitoring 

■ Knowledge-generating products 

■ Identification and replication of 
promising practices 

■ Technical assistance assessment 


■ Guidelines for high-quality evaluation 
designs (process and outcome) 

■ Logic models for program and evaluation 
designs 

■ List of evaluation measures with 
instrumentation 

■ Guidelines for evaluation products 


GRANTEES: 
STUDY SITES 


■ Grant applications 

■ Project development, implementation 

■ Local evaluation management 

■ Local evaluation coordination 

■ Knowledge-generating product 
development 


■ Evaluation plan outline 

■ Process and outcomes evaluation designs 

■ SDU, clinician, cost, and client measures 

■ Roles and responsibilities for grantee 
provider/evaluator staff 

■ Guidelines for evaluation products 


GRANTEES: 

MULTI-SITE 

EVALUATORS 


■ Grant applications 

■ Comprehensive evaluation designs 

■ Evaluation implementation: 

- Data collection 

- Data analysis 

- Reporting evaluation findings 

■ Evaluation product development 


■ Evaluation concepts 

■ Logic models 

■ Evaluation designs 

■ Evaluation data requirements 

■ Data collection instrumentation 

■ Data collection process and procedures 

■ Data analysis 

■ Product development 


NATIONAL 

EVALUATORS/ 

SERVICES 

RESEARCHERS 


■ Contract applications 

■ Comprehensive evaluation designs 

■ Evaluation implementation: 

- Data collection 

- Data analysis 

- Reporting evaluation findings 

■ Evaluation product development 


■ Evaluation concepts 

■ Logic models 

■ Evaluation designs 

■ Evaluation data requirements 

■ Data collection instrumentation 

■ Data collection process and procedures 

■ Data analysis 

■ Product development 



IEM products and other evaluation materials may be obtained from: 

http://neds.calib.com « » , a • i a ni C 

best copy available 
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Editor’s Note 



This document is one of a series of papers that describe CSAT’s approach to substance 
abuse treatment evaluation. The graphic below illustrates the continuous evaluation knowledge 
development and application process which characterizes CSAT’s approach. At the core is the 
self-adjusting treatment evaluation model which is the foundation. The model integrates 
continuous, state-of-the-art evaluation with plaiming, management, operation, and service 
delivery within a multi-disciplinary learning community. Implementation of this model requires 
building of team capabilities, appropriate, state-of-the-art performance evaluation and cost 
analysis, and assimilation of CSAT’s integrative approach to treatment evaluation and integrative 
methodologies. Each of these processes work together to ensure continuous improvement. 



Ensuring Continuous Evaluation Knowledge Development and 

Application 




Assimilate Approach to ^ w " Ensuring Continuous, State-of-the-Art ^ ^ Plan and Conduct 

Treatment Evaluation Evaluation ^ Performance Evaluation 
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Substance abuse treatment providers are increasingly called upon to demonstrate that they 
are delivering appropriate services, that those services have the desired impact, and that the 
services justify the costs. An ongoing process of evaluation and systems/services improvement 
integrated into the day-to-day operation of treatment providers is needed to do so. In addition, 
the evaluation and improvement process requires a multi-disciplinary team that includes 
treatment personnel, evaluators. Federal and State agencies, advocacy groups, funding agencies, 
and the community. Building team capability is integral to this approach. Treatment staff must 
be involved in knowledge development and application (i.e., planning and implementing 
evaluation efforts, incorporating changes in response to new knowledge, and sharing of 
findings). 
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